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(57)Abstract: 

PURPOSE: To enable a processing at a real time by 
extracting a moving object by processing an input 
image, detecting the attribute of the object, labelling 
it and measuring the number of moving objects by 
attributes. 

CONSTITUTION: At a moving object extraction part 
1-1, a moving object area in the image is calculated 
by successively inputting moving images containing 
moving objects. Next, at a moving object area 
separation part 1-2, the moving object area is 
segmented as a single image at every linked area in 
the extracted moving object group and stored in an 
area data memory 1-3. Further, an image recognition 
processing part 1-5 performs attribute judgement 
and counting to the extracted area image separated 
to be an object allocated by a schedule managing 
part 1-4 corresponding to processing process for judgeing two or more attributes. 
Thus, the moving object image can be recognized at a high speed and high level 
concerning the moving image provided from a TV camera or the like. 
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CLAIMS 
[Claim(s)] 

[Claim 1] The animal object extract process in which an animal object field is extracted 
from an input image, and the animal object field separation process which carries out 
sequential separation of this extract field, and is accumulated in field data memory with 
the separated time of day, In the image recognition processing process which consists 
of at least two or more treatment processes which detect the attribute of the 
separated animal object, and said attribute detection The schedule management 
process in which advance of said treatment process is managed in the direction in 
which the number of attribute judgings per unit time amount increases, The animal 
object recognition art using the multi-process characterized by having the integrated 
processing process which unifies the result of said treatment process for every unit 
time amount, and carries out labeling to counting or this animal object for every 
attribute of an animal object. 

[Claim 2] An input image is an animal object recognition art using the multi-process 
according to claim 1 which is an image between slit space-time and is characterized by 
performing an extract by taking and making difference with a background image binary 
in an animal object extract process. 

[Claim 3] An input image is an animal object recognition art using the multi-process 
according to claim 1 which is an image between slit space-time and is characterized by 
performing an extract by performing a correlation operation with a background image in 
an animal object extract process, and performing threshold processing. 
[Claim 4] Two or more processes of using for an image-recognition processing process 
are the animal object recognition arts using the multi-process according to claim 1 
characterized by to consist of a treatment process A group which performs directly 
easy attribute judging processing to the extract field separated according to the animal 
object field separation process, and a treatment process B group which performs 
comparatively complicated attribute judging processing to the image once accumulated 
in field data memory. 

[Claim 5] A treatment process B group is an animal object recognition art using the 
multi-process according to claim 4 characterized by performing processing to an 
animal object field in the order which constituted in juxtaposition two or more image 
recognition treatment processes which perform the same processing, and was assigned 
by the schedule management process. 

[Claim 6] A treatment process B group is an animal object recognition art using the 
multi-process according to claim 4 characterized by performing processing which 
constitutes the image recognition treatment process which performs different 
processing in or more at least one juxtaposition, and has a processing demand 



according to a schedule management process in the order arranged for every 
processing kind. 

[Claim 7] The need for prediction or processing be judge for the throughput of an 
attribute judging for every separated extract field , and a process be specify as it , and 
a processing image and a recognition result be re-accumulate after predetermined 
processing progress , and a schedule management process be an animal object 
recognition art using the multi-process according to claim 1 characterize by 
dissociate , or accumulate with recognition data respectively , without re-dissociate 
re-about a field image . 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application] Using the image processing by the multi-process, this invention 
recognizes the attribute of a migration body (for example, people, a car, the body on a 
conveyor belt), and relates to the animal object recognition art which performs labeling 
or processing which carries out counting according to this attribute. 
[0002] 

[Description of the Prior Art] An animal object is extracted out of a time series image, 
and the attempt which carries out statistics processing of the data according to the 
attribute of the extracted animal object is advanced. It is carrying out counting 
according to a passerby's attribute from the image which specifically photoed the 
passerby who walks along a foot walk etc. Here, attributes are that he is human being, 
sex, age, a dress, a dress color, passing speed, the migration direction, etc. 
[0003] In such counting, on-line processing is desirable, the attribute exception which 
analyzed the dynamic image obtained with frame spacing extent, and has unified and 
recognized data to be on-line processing here for every unit time amount for several 
seconds thru/or about 10 minutes — counting — it is obtaining a result, that is, — 
although an input image is the real time (dozens of times per second) — counting — a 
result means processing made into several seconds - every 10 minutes according to 
the use purpose. 

[0004] If counting of the passerby can be carried out according to an attribute in such 
on-line processing, various use, such as visitor measurement in traffic measurement in 
a shopping center, a store, and the event hall, can be considered. 
[0005] 

[Problem(s) to be Solved by the Invention] When it aims at the above use, the trouble 
of a Prior art is as follows. 

(1) the algorithm which extracts and carries out counting of the animal object 
according to an input image — various **** — the attribute of an animal object — 
subdividing — counting — the algorithm which processes a result statistically is not 
realized. 

(2) The single configuration of a process is common. That is, an animal object is 
extracted from an input image and the method of processing by single-loop processing 
until it carries out counting of this is taken. In this case, the complexity of the attribute 
judging processing of an animal object in which a system is possible will be decided 
with the most complicated input image. That is, when it is going to process the input 
image of all situations in the real time, it is necessary to decide by the design stage 
what kind of attribute judging is performed for the most complicated image. 



[0006] a complicated situation which people's herd will pass if a passerby's counting is 
taken for an example — taking into consideration — counting — when it is going to 
operate a system normally, it is difficult to judge and carry out counting of a passerby's 
fine attribute in respect of the processing time, and is only carrying out counting of the 
passerby according to a direction at most. 

(3) If complicated processing is performed in an input image with an eye on 
improvement in counting accuracy etc., in order to exceed 33msec(s) in many cases, 
real-time requirement is lost. 

(4) Although the attempt which takes in a multi-process to an image processing is 
starting recently, there is almost no example which recognizes the attribute of a 
migration body finely and carries out statistics processing for every unit time amount. 
Furthermore, there is also no example using DS required for this or an integrated 
processing algorithm. 

[0007] As mentioned above, conventionally effective technique is not proposed to 
attribute recognition processing of the migration body in the online which this invention 
makes an aim. This invention aims at making processing possible in the real time. 
[0008] 

[Means for Solving the Problem] this invention — set — (1) input image — the real 
time — incorporate — and counting — (3) to which the animal object which extract an 
animal object so that a leak may not arise, and which carried out (2) extracts detect an 
attribute in a detail as much as possible — for every unit time amount for several 
seconds - about 10 minutes, counting of the migration body be carry out according to 
an attribute, or an attribute identifier (label) be give to a migration body (on-line) — it 
be make like. 
[0009] 

[Function] And this invention processes an input image, extracts an animal object, 

detects the attribute of this body, and it carries out labeling to this body, and it 

measures the number of animal objects according to this attribute. 

[0010] Specifically an animal object field image is separated and accumulated with 

separation time of day for every connection field, real-time processing of each 

connection field group which is the schedule which considers the throughput and the 

need for processing as decision description, and was accumulated is carried out by the 

image recognition treatment process constituted by multistage and juxtaposition, and 

image recognition processing of the variety of each field is performed, every 

[ furthermore, ] field — integration of all recognition results — carrying out — after 

unit time amount progress — an attribute exception — counting — a result is 

outputted. 

[0011] 

[Example] When this invention is summarized in advance of explanation, it is a thing 
like a degree, separating an animal object field, accumulating in memory with separation 
time of day in claim 1, and processing this in two or more processes — image 
incorporation, attribute recognition, and counting of the real time — since processing 
is independently controllable, it can respond to change of an input image flexibly. That 
is, counting according to attribute is made, holding real-time requirement also to a 
complicated image. 

[0012] claim 2 — setting — not a two-dimensional image but a slit image — using — 
moreover, difference — it is performing binary-ization and high-speed processing is 
attained. In claim 3, it is effective in improvement in the robustness in the lighting 
fluctuation in the case of an extract by using a correlation operation. That is, even 



when the brightness of the background which exists in a processing-object image 

shifts compared with a reference background image by change of a sunshine condition 

etc., it becomes possible to extract an animal object field correctly. 

[0013] In claim 4, even when time is taken very much for example, in attribute judging 

processing and a process is overdue by dividing a configuration process into two 

groups according to the processing purpose, an easy attribute judging is surely 

performed and counting of low precision can be performed. 

[0014] In claim 5, a schedule can be set up so that a processing load may be 

distributed and each process may always process, and it becomes possible to perform 

an attribute judging at a high speed. 

[0015] In claim 6, required sufficient processing can be performed to the target animal 
object field, and more nearly high-speed processing also becomes possible by 
parallelization of the same treatment process further. 

[0016] In claim 7, useless processing cannot be performed to an object domain and 
multiple processes can be used efficiently. Furthermore, according to a case, more 
detailed recognition processing is attained by re-division of an animal object field. 
[0017] The technique of performing hereafter the extract of the highly precise 
counting and the dress color attribute of a person image which walk on a foot walk as 
one example of this invention is explained with reference to the block diagram of 
drawing 1 . In addition, it is easy an object to apply to another migration body instead 
of a person, and it is easy to be made to extract other attributes. Also by the signal of 
the distance data based on a supersonic wave, infrared radiation, etc., a sample signal 
is possible, and should just read and replace brightness with a distance value. 
Hereafter, it explains as an example of the TV camera input under the light. 
[0018] First, in the animal object extract section 1-1, the dynamic image containing a 
migration body is inputted serially, and it asks for the animal object field in an image. 
That is, the image information on one line equivalent to measurement Rhine samples, a 
1 -dimensional slit image obtains from the image which photoed the person who passes 
through measurement Rhine on the foot walk which is a background by the fixed angle 
type, and after extract the part which is changing to a background image and 
performing the noise component removal by the plastic-surgery processing on Rhine, 
and separation of a shadow field if needed, a slit-like change field image and a 
corresponding real image accumulate. 

[0019] However, not only a TV camera but a 1 -dimensional sensor may be used for 
acquisition of a 1 -dimensional slit image, and a background image performs renewal of 
a background using the input image to current if needed using a slit image or the 
background image fixed beforehand when the time of there not being an initial frame or 
a pedestrian is chosen at its own discretion and a user takes out a command with the 
time of initiation of operation. 

[0020] Here, an animal object extract according to claim 2 calculates the absolute 
value of the difference of an input image to a background image, and is realized by 
binary — ization-extracting the part which is changing and making a binary-ized slit 
image. 

[0021] Moreover, an animal object extract according to claim 3 performs the 
correlation operation of a background image and an input image to each pixel on a slit, 
and it calculates, similarity, i.e., a correlation value, with a background image. 
Furthermore, it realizes by making the binary-ized slit image which judges the pixel 
which carries out threshold processing of the correlation value of each pixel, and is 
changing, and includes an animal object field. 



[0022] Next, in the animal object field separation section 1-2, it starts as a single 
image for every connection field of animal **** extracted by the animal object extract 
section 1-1, and accumulates in the field data memory 1-3. That is, labeling of the 
change field image obtained in order in the real time is serially carried out to a target, 
and a series of connection field images used as the same label are formed. 
Furthermore, the connection field image is used for a mask, superposition and a two- 
dimensional animal object field image are made for the information on a real slit image, 
and both are accumulated in the field data memory 1-3 for every field. 
[0023] Moreover, characteristic quantity, such as 0, the 1 or secondary moment, and a 
minimax value of a field coordinate, is calculated about each connection field image to 
coincidence, and it accumulates in the field data memory 1-3 with the separation time 
of day. Here, in the field data memory 1-3, a connection field image and an animal 
object field image, the separated time of day, characteristic quantity, and a recognition 
result are accumulated for every field. Data are outputted and inputted with directions 
of the schedule Management Department 1-4 to the image recognition processing 
section. 

[0024] At the schedule Management Department 1-4, it manages so that processing 
with the number of attribute judgings sufficient and required per unit time amount may 
have an attribute judging treatment process to the animal object field data serially 
generate by the target by the above-mentioned processing advance. 
[0025] Here, at the schedule Management Department according to claim 7, the array 
and assignment of the optimal field to each attribute judging process are performed, 
and each accumulated connection field is inputted into each process of the image 
recognition processing section so that the need for prediction of the throughput which 
changes with contents of processing, and various processings is judged for every 
connection field, a processing load is distributed, each process may always process 
and only required processing may be performed. Furthermore, the processing image 
and recognition result after each process termination are re-accumulated. Moreover, a 
field image is re-separated if needed and it accumulates with recognition data 
respectively. 

[0026] Next, in the image recognition processing section 1-5, the treatment process 
which performs at least two or more attribute judgings performs attribute judging and 
counting to the separated extract field image used as the object assigned by the 
schedule Management Department 1-4. 

[0027] Here, the image recognition processing section according to claim 4 consists of 
a treatment process A group 1-6 which performs directly easy attribute judging 
processing, and a treatment process B group 1-7 which performs comparatively 
complicated attribute judging processing to the image once accumulated in the field 
data memory 1-3. 

[0028] ,s such as field division using brightness information as a treatment process, 
field division using color information, matching with profile information and the model 
image in a pedestrian's image between space-time, and person head detection by the 
person model, can be considered. 

[0029] Distinguishing a noise field and the field equivalent to solo passage by the 
treatment process A group 1-6 based on characteristic quantity, such as 0, the 1 or 
secondary moment, and a minimax value of a field coordinate, about each connection 
field accumulated in the field data memory 1-3 counting — the passing direction is 
judged by calculating the number on the basis of a solo passerby's datum-level product 
value from the area value more than a threshold, and calculating the grade of a field if 



needed. 

[0030] A treatment process B group according to claim 5 constitutes in juxtaposition 
two or more image recognition treatment processes which perform the same 
processing, and performs processing to an animal object field in the order assigned by 
the schedule Management Department 1-4. 

[0031] For example, like drawing 2 , an animal object field is extracted in order at each 
time of day (A, B, C, D, E), and presupposes that the throughput of each field to a 
certain attribute processing a is predicted (suppose that the same is said of the 
throughput to other attributes). In addition, the alphabetic character of the left end in 
the oblong rectangle shown in drawing 2 thru/or drawing 6 expresses a field label, and 
the rectangular die length corresponds to throughput. What is necessary is just to take 
a processing schedule like drawing 3 f in order to process each field so that a 
processing load may be distributed and the number of attribute judgings per unit time 
amount may increase when the processes a (for example, counting by field area 
processing, field division processing using color information, etc.) which perform the 
same processing like drawing 3 are constituted in three-piece juxtaposition at this 
time. Moreover, what is necessary is just to advance processing to the same 
processing schedule like drawing 4 , even if it regards it as one treatment process X 
combining each attribute judging processing to a serial in order to perform a more 
complicated attribute judging. 

[0032] Moreover, a treatment process B group according to claim 6 constitutes the 
image recognition treatment process which performs different processing in plurality 
[ every ] juxtaposition, and performs processing which has a processing demand by the 
schedule Management Department in the order arranged for every processing kind. 
[0033] For example, what is necessary is to constitute in juxtaposition the processes 
a, b, c, and d (for example, the field division processing using brightness information, 
field division processing using color information, person head detection by the person 
model, etc.) which perform different attribute extract processing like drawjng 5 , to 
arrange to a treatment process and just to advance processing in the extract situation 
of an animal object field like drawing 2 , when judged with there being the need for 
processing by the schedule Management Department about each field. Moreover, what 
is necessary is to combine each attribute judging processing with juxtaposition, and 
just to assign a field group with the need for processing to a process like drawing 6 , in 
the direction whose number of attribute judgings per unit time amount increases in the 
corresponding field group, in order to perform a more nearly high-speed attribute 
judging. 

[0034] furthermore, counting which performs each attribute judging and counting in the 
integrated processing section 1-8 using two or more sorts of processing results 
obtained by random from the image recognition processing section 1-5 about each 
connection field, unifies after unit time-amount progress (after that — every unit time 
amount), and can be set by the present time of day — a result is outputted and 
corrected and it outputs to the data-output section 1-9. for example, counting [ in / 
advanced counting is performed from the field division result using color information 
and head detection result of an object domain, and / a treatment process A group ] — 
a result is corrected. Moreover, statistics processing of the color field in a field is 
performed from a field division result, and a person's dress color attribute is judged. 
[0035] the last — the data output section 1-9 — the attribute judging and counting by 
the present time of day — a processing result is outputted. 
[0036] 



[Effect of the Invention] With the conventional technique, since processing was 
advanced in the single process in order and the processing result was serially 
outputted to the motion area which processes a dynamic image for every unit time 
amount, and is obtained, the object domain of processing could be complicated, or in 
order to make more advanced recognition processing perform, when the complicated 
image processing was performed, processing took time amount, and the real-time 
requirement of processing was not able to be maintained. 

[0037] According to this invention, there is an advantage that the advanced migration 
body image which is a high speed can be recognized from the dynamic image obtained 
from images, such as a TV camera. 
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DESCRIPTION OF DRAWINGS 
[Brief Description of the Drawings] 

[Drawing 1] It is a block diagram explaining the animal object recognition approach. 
[Drawing 2] It is drawing explaining the throughput predicted to be the extract situation 
of a certain animal object field. 

[Drawing 3] It is drawing explaining parallel form processing of the same treatment 
process. 

[Drawing 4] It is drawing explaining the parallel form processing by the treatment 
process considered as the serial configuration of a uniprocessing process. 
[Drawing 5] It is drawing explaining parallel form processing of a different-species 
treatment process. 

[Drawing 6] It is drawing explaining the processing made into two or more parallel 

forms of a different-species treatment process. 

[Description of Notations] 

1-1 Animal Object Extract Section 

1-2 Animal Object Field Separation Section 

1-3 Field Data Memory 

1-4 Schedule Management Department 

1-5 Image Recognition Processing Section 

1-6 Treatment Process A Group 

1-7 Treatment Process B Group 

1-8 Integrated Processing Section 

1-9 Data Output Section 
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